United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



I ATTORNEY DOCKET NO. [ CONFIRMATION NO. j 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



10/067,562 



02/05/2002 



02/10/2006 



Bertrand Chupeau 



7590 

JOSEPH S. TRIPOLI 

THOMSON MULTIMEDIA LICENSING INC. 
2 INDEPENDENCE WAY 
P. O. BOX 5312 
PRINCETON, NJ 08543-5312 



PF0 10009 



5056 



EXAMINER 



3 



LAROSE, COLIN M 



ART UNIT 



PAPER NUMBER 



2627 

DATE MAILED: 02/10/2006 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

10/067,562 


Applicants) 
CHUPEAU, BERTRAND 


Examiner 

Colin M. LaRose 


Art Unit 

2627 





•• The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• if NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )EE3 Responsive to communication(s) filed on 16 December 2005 . 
2a)D This action is FINAL. 2b)[3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) I3 Claim(s) 1-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-8 and 19 is/are rejected. 

7) H Claim(s) 9-18 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 12 May 2005 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 1 1 9 

12)13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
bM A" b)Q Some * c)D None of: 

1 .M Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) □ Notice of References Cited (PTO-892) . 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(syMail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date . 6) O Other: . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 7-05) 



Office Action Summary 



Part of Paper No./Mail Date 0206 



Application/Control Number: 10/067,562 Page 2 

Art Unit: 2627 

DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1 .17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 16 December 2005 has been entered. 

Claim Objections 

2. In view of Applicant's amendments, the previous claim objections are withdrawn. 

3. Claims 9, 11, 15, and 17-19 are objected to because of the following informalities: 
In claim 9, the variable S re should be defined, for example by placing "grouping 

threshold" before the first occurrence thereof; 

In claim 1 1, the calculation of Apj/i appears to be incorrect in that the first parentheses 
should not contain three terms; 

Claim 15 should conclude with a period; 

In claim 17, "the previous image" should be changed to "a previous image" since it is the 
first occurrence thereof in the claims; 

In claim 18, the claimed equation should be rewritten so that its appearance is accurate; 
In claims 18 and 19, the "we use" language should be removed; and 
In claim 19, the period after "Claim" should be removed. 
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Since this application was not originally written in English, Applicant is respectfully 
requested to review the claims and correct any idiomatic or grammatical errors contained therein. 
Appropriate correction is required. 

Response to Amendments and Arguments 
4. Applicant has amended claim 1 to denote that C is a "factor consistent with the inverse of 
a mean number of pixels" and argues that Haris does not disclose, "C being consistent with the 
inverse of a mean number of pixels," as claimed. In the Examiner's opinion, this limitation is 
insufficient to distinguish the claimed invention from the Haris reference. 

As stated in the prior Office action, Haris discloses C being equivalent to the inverse of 
(||R\|| + ||R\||), which is the sum of the cardinality of fragments Ri and Rj. The cardinality of a 
set is simply the number of elements in the set, therefore (HR'mII + i|R* j M||) represents the 
aggregate number of pixels in the two fragments. 

Applicant argues that this inverse of the aggregate number of pixels is not "consistent 
with the inverse of a mean number of pixels " Examiner respectfully disagrees. The mean value 
of elements between two groups of elements is equivalent to the aggregate number of elements 
divided by two. Therefore, the aggregate value is equal to the mean value multiplied by a known 
scalar value. The aggregate and mean values are considered to be consistent with each other 
because they are directly proportional with each other by a factor of two. 

The Specification states, "C is consistent with the inverse of a number of pixels" 
(paragraph [0100]). Then in the next paragraph, the Specification describes how preferably C is 
the mean value of the two fragments Fi and Fj (paragraph [0101]). The Specification suggests 
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that C being "consistent with a number of pixels" means that the value C is directly proportional 
to a number of pixels and thus may include the mean value of a number of pixels. The converse 
is also true in that the mean number of pixels is consistent with the aggregate number of pixels. 
A suggested correction is to denote that "C is the inverse of a mean number of pixels" rather than 
C being merely consistent with a mean number of pixels. Such a change would at least overcome 
the current rejection of claim 1 under 35 USC § 102(b). 

Applicant presents the same arguments for the other claims, and the above remarks are 
applicable to those claims. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

6. Claims 1-6 are rejected under 35 U.S.C. 102(b) as being anticipated by "Hybrid Image 
Segmentation Using Watersheds and Fast Region Merging" by Haris et al. ("Haris"). 

Regarding claim 1, Haris discloses a method for the fragmentation of images into 
homogeneous regions, this fragmentation using iterative merges of fragments Fi and Fj (i.e. two 
regions R*m and R* j M) which are as similar as possible according to at least one selection 
parameter, this similarity being evaluated by a product A*B of two factors A and B (see equation 
(12), page 1689), A being consistent with a number of pixels (A = ||R*m|| * I|R**m|| ) and B being 
consistent with the selection parameters (B = [n(||R\f||) - \i (||R\i||)3 2 ), characterized in that a 
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merge is performed when the product A*B*C is less than a threshold consistent with the 
selection parameter, C being a factor consistent with the inverse of a mean number of pixels (C = 
1 / (||R\i|| + ||R* j M||), which is directly proportional to the inverse of the mean number of pixels 
and is therefore consistent the inverse of the mean number of pixels). 



Regarding claim 2, Haris discloses that C is proportional to 2/N m where N m represents a 
mean size of the fragments Fi, such as a mean number of pixels (i.e. C represent the inverse of 
the sum of the cardinality of sets R*m and R^m and is proportional to the inverse of the mean 
number of pixels). 

Regarding claim 3, Haris discloses the selection parameter(s) is (are) at least luminance 
(i.e. the luminance Y is used in the calculation of B). 

Regarding claim 4, Haris discloses that each iterative merge relates only to two 
neighbouring fragments Fi and Fj ( 7(i,j) in equation (12) shows that the merge only relates to 
adjacent regions). 

Regarding claim 5, Haris discloses that the factor A is proportional to the claimed 
equation (i.e. A = ||R\i|| * ||R**m||, which is proportional to the given equation). 

Regarding claim 6, Haris discloses that the size of a fragment is the number of pixels 
included in this fragment (i.e. the size of each region is the cardinality of the region - that is, the 
number of pixels in the region). 
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Claim Rejections - 35 USC§103 

7. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. Claim 7 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Harris in view of . 

Regarding claim 7, Haris discloses that the factor B is proportional to (Yj - Yj) 2 , where 
(Yi - Yj) represent the difference between the luminances of each of these two fragments (see 
equation (12), page 1689). 

Haris does not disclose that the U and V components are utilized because Haris' 
disclosure is in terms of a greyscale image. 

Moed discloses an image segmentation system that operates on a color image. In 
particular, Moed teaching utilizing the YUV color format to segment an image, and the 
segmentation algorithm involves generating Y, U, and V difference information. See column 4, 
lines 18-48. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Haris by Moed to segment a color image in the YUV format and utilize the squared 
difference of the U and V channels, since Moed discloses that it is conventional to perform color 
segmentation on YUV images and since color images are utilized in a wide variety of 
applications since they are generally more pleasing to the eye than greyscale images. 
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9. Claims 8 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Matching Error Based Criterion of Region Merging for Joint Motion Estimation and 
Segmentation Techniques" by Schutz et al. ("Schutz") in view of Haris. 

Regarding claim 8, Schutz discloses a method of grouping fragments of an image (i.e. 
regions), characterized in that this grouping uses a model of motion individual to each fragment 
Fi (§ 3 - the motion M of each region is estimated and used to group, or merge, the regions 
together), this model of motion is determined with a known error or variance so as to allocate to 
the fragment created by this grouping one and the same model of motion (§ 4 - the motion of 
model for each grouped (merged) region is determined with the known mean absolute error 
(MAE) of the regions previously computed), grouping together at each step of this method the 
fragments Fi and Fj which minimize a grouping cost proportional to an evaluation of the 
difference between the models of motion of the two fragments Fj and Fj (§ 3 - the difference in 
the motion models, Ma and Mb, of two regions, A and B, is determined on the basis of the mean 
absolute error (MAE); § 4 - the MAEs between each pair of regions are evaluated to determine 
which adjacent regions exhibit the lowest merging cost, and then those regions whose grouping 
costs were determined to be smallest are merged). 

Schutz discloses that the initial regions are obtained by a spatial watershed segmentation 
(see § 2). 

However, Schutz does not disclose that the initial fragments (regions) are obtained 
according to the method of claim 1 . 
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Haris discloses the method of spatial segmentation according to claim 1. It would have 
been obvious to one of ordinary skill in the art at the time of the invention to modify Schiitz by 
Haris to create the initial segmentation according to claim 1, since Schiitz discloses that the 
initial segmentation of the regions is accomplished through a spatial watershed segmentation 
algorithm, and Haris shows that a watershed segmentation that employs the details of claim 1 is a 
conventional way of spatially segmenting an image into homogeneous regions and achieves 
satisfactory results for such a purpose. 

Regarding claim 19, Schiitz discloses the stopping condition is given by a minimal 
threshold (see § 4). 



Allowable Subject Matter 
1 0. Claims 9- 1 8 are objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

Regarding claim 9, Schiitz does not disclose: (1) that the grouping cost is inversely 
proportional to a threshold of motion, or (2) that in the absence of any other stoppage test, the 
grouping is not allowed if the evaluation is greater than this threshold. Rather, Schiitz teaches 
determining the grouping costs between each pair of adjacent regions and merging regions that 
have a minimal cost (see § 4). 

Regarding claim 1 1, Schiitz does not disclose the claimed details of the delta equations 
that are used to calculate the evaluation of the difference of motion between two fragments. 
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Regarding claim 18, Schutz is silent to the claimed stoppage test formula. 



Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (571) 272-7423. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Jingge 
Wu, can be reached on (571) 272-7429. The fax phone number for the organization where this 
application or proceeding is assigned is (571) 273-8300. Any inquiry of a general nature or 
relating to the status of this application or proceeding should be directed to the TC 2600 
Customer Service Office whose telephone number is (571) 272-2600. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Colin LaRose vyV* 
Group Art Unit 2627 
3 February 2006 




